Search for: All records

Along with a number of other computing technologies, cloud computing services are increasingly being promoted as a way of enabling openness, reproducibility, and the acceleration of scientific work. While there have been a variety of studies of the cloud in terms of computing performance, there has been little empirical attention to the changes going on around cloud computing at the level of work and practice. Through a qualitative, ethnographic study, we follow a cosmology research group’s transition from a shared high performance computing cluster to a cloud computing service, and examine the cloud service as a coordinative artifact being integrated into a larger ecology of existing practices and artifacts. We find that the transition involves both change and continuity in the group’s coordinative work and maintenance work, and point out some of the effects this adoption has on the group’s larger set of practices. Finally, we discuss practical implications this has for the broader adoption of cloud computing in university-based scientific work. 

The growth in scientific data volumes has resulted in a need to scale up processing and analysis pipelines using High Performance Computing (HPC) systems. These workflows need interactive, reproducible analytics at scale. The Jupyter platform provides core capabilities for interactivity but was not designed for HPC systems. In this paper, we outline our efforts that bring together core technologies based on the Jupyter Platform to create interactive, reproducible analytics at scale on HPC systems. Our work is grounded in a real world science use case - applying geophysical simulations and inversions for imaging the subsurface. Our core platform addresses three key areas of the scientific analysis workflow - reproducibility, scalability, and interactivity. We describe our implemention of a system, using Binder, Science Capsule, and Dask software. We demonstrate the use of this software to run our use case and interactively visualize real-time streams of HDF5 data.